Move CloudBatchSubmitJobOperator job normalization out of __init__ - #71201
Open
yunusakca wants to merge 1 commit into
Open
Move CloudBatchSubmitJobOperator job normalization out of __init__#71201yunusakca wants to merge 1 commit into
yunusakca wants to merge 1 commit into
Conversation
Template fields are rendered after the constructor runs, so any read of a template-field value inside __init__ operates on the un-rendered Jinja expression rather than the real value. CloudBatchSubmitJobOperator's job protobuf-to-dict conversion (added for apache#37217, so the template renderer can descend into nested fields) is exactly that pattern. The conversion has to happen before Jinja rendering, not after, or nested Jinja expressions inside a protobuf Job would never be discovered by the renderer. Move it into prepare_template(), the hook Airflow already runs after construction but before rendering — mirroring the same pattern used by CloudBuildCreateBuildOperator in this provider. Part of the exemption-list burn-down tracked in apache#70296.
yunusakca
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl,
potiuk and
shahar1
as code owners
August 5, 2026 19:35
boring-cyborg
Bot
added
area:dev-tools
area:providers
backport-to-v3-3-test
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves
CloudBatchSubmitJobOperator'sJobprotobuf → dict normalization out of__init__, as part of the exemption-list burn-down tracked in #70296.Template fields are rendered after the constructor runs, so reading a template
field's value inside
__init__operates on the un-rendered Jinja expressionrather than the real value. The
jobprotobuf-to-dict conversion — added for#37217 so the renderer can descend into nested fields such as a runnable's
container commands — is exactly that pattern.
The conversion has to run before Jinja rendering, not after: moving it to
execute()would silently stop nested Jinja expressions inside aJobprotobuf from ever being rendered. It now lives in
prepare_template(), thehook Airflow runs after construction but before rendering, matching the pattern
CloudBuildCreateBuildOperatoralready uses in this provider.CloudBatchSubmitJobOperator's entry is removed fromvalidate_operators_init_exemptions.txt.The existing rendering test already covers both a
dictand a protobufJobinput; a new test asserts the normalization happens in
prepare_template()rather than the constructor.
related: #70296
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Sonnet 5 and Opus 5) following the guidelines